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Abstract 

Using Error Vector Magnitude (EVM) in satellite integration and test allows rapid 
verification of the Bit Error Rate (BER) performance of a satellite link and is particularly 
well suited to measurement of low bit rate satellite links where it can result in a major 
reduction in test time (about 3 weeks per satellite for the GOES satellites during ground test) 
and can provide diagnostic information. Empirical techniques developed to predict BER 
performance from EVM measurements and lessons learned about applying these techniques 
during GOES N, O, and P integration and test and post launch testing, are discussed. 


I. Introduction 


The Geosynchronous Operational Environmental Satellite (GOES) N, O, and P satellites, while primarily weather 
and science satellites, also have eight specialized communication links. Each link is unique and varies in data rate 
from 100 bps to 2.66 Mbps. Two links use uncoded RF links to transmit data generated on the satellite. Unlike 
BER measurements, EVM allows the BER to be determined with actual data, which might be different from a 
Pseudo Random (PN) code test pattern. These characteristics make EVM measurements the best method to verify 
link performance without taking the link out of service to test with a test pattern. 
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EVM is a well established technique used in 
measuring performance of cellular communication 
systems 1 ' 2 . The application of EVM to satellites is 
not new, as it has been used to characterize unit 
level 3 and system level performance in Europe and in 
the US, specifically at NASA 4 . Consequently, 
commercial vector signal analyzers (VSAs) to 
measure EVM are readily available from major 
electronic companies including Agilent, Tektronics, 
and Rhode&Schwarz. 

EVM is simply the standard deviation of the points 
about an average value of a constellation point 
(Figure 1). The constellation plot provides 
information on the link quality and, unlike a BER 
measurement, provides information about the factors 
impairing the link that can be used as diagnostic 
information about the degradations that limit 
performance. EVM can measure performance on an 
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Figure 1 Measured EVM for BPSK. EVM shows 
difference between same bit transmitted (squares) 
and a change in bit (x's) 
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operational link that contains all impairments up to the EVM measurement point (which can be RF or IF). A second 
major advantage of EVM over BER tests is that knowledge of the original bit sequence is not needed. This was 
particularly important for the GOES N, O, & P satellites as they have science data generated on-board the satellite 
that is transmitted without coding to the ground. Figure 1 also shows that the constellation plot is sensitive to its 
history due to intersymbol interference (due to the VSA itself, channel filter mismatch, ringing, or other effects). If 
the bit is the same as the previous bit, its constellation point is different than if the bit has changed. This give rise to 
the two blob effect discussed later. 

Some authors have said that EVM cannot be used to predict BER. For example Johnson (2001) states, “EVM is a 
summation of effects. It does not differentiate between random effects which add geometrically and possess 
probability densities, from distortion effects which add directly and are deterministic. EVM is useful because its 
characteristics can tell us something about the nature of the problem or where the signal is degraded, but it can not 
be directly related to BER, the acid test for the network operator.” 5 Here we show that EVM can be used to 
determine BER using an empirical calibration. Once a calibration curve is obtained, a very quick EVM 
measurement (typically 60,000 samples) can be used to determine BER to very low bit rates. This was especially 
useful for low data rate signals, such as the 1 kbps and lower rates on the GOES satellites. EVM can be used to 
predict BER only if the changes are due to changes in the AWGN (additive white Gaussian noise). If the change in 
BER is due to another mechanism, perhaps an increase in phase noise, the technique developed here will show that a 
change has occurred, but its prediction of the BER associated with the change may not be accurate. 

II. Relating EVM to BER 

While in principle the BER could be calculated solely from the standard deviation of the EVM, such calculations do 
not agree with BER measurements in part because they do not take into account modem impairments. It was 
necessary to develop equations to relate BER (or probability of error. Pet) to EVM as shown below 6 . The equations 
developed have two variables a and 
P which are adjusted to give a best 
fit to the measured BER curve (bit 
error rate vs. signal to noise ratio). 

This measured curve is obtained 
from simultaneous BER and EVM 
measurements (or separate 
measurements under the same 
conditions). The number of samples taken determines the accuracy of the measurement. About 60,000 
measurements provide an EVM value with a standard deviation accurate to about 0.4%. (Please refer to the 
companion paper 7 for the derivation of the formula shown here and for more information on development of the 
equations, variables and measurement methods). 

Figure 2 shows a measured BER curve and the best fit curve, where a and P were adjusted for best fit. This 
agreement is typical for the GOES measurements. Another useful aspect of EVM is demonstrated in Figure 2. The 
performance at extremely low BERs can be extrapolated backward. The link shown in Figure 2 is a 25 kbps link. 
What would normally be a very time consuming measurement, determining BER to 10' 8 requiring approximately 
10 9 bits or about 10 hours can be done in minutes with EVM. 

III. Intersymbol interference and the "Two-Blob Effect" 

Non-ideality in the VSA, the RF modulator or filtering can result in inter-symbol interference (ISI) s . For example 
most VSA’s have a stock set of matching filters (e.g. Raised Cosine, Root Raised Cosine), and if none truly match 
the spacecraft hardware then ISI will result, as shown in figures 1 and 3. 

Using a simple EVM calculation would result in a distorted EVM value as it would imply an oval shape. Since we 
are using EVM to calibrate the AWGN response of a system, we need to remove the effect of the ISI in order to 
isolate the AWGN component of EVM. This was accomplished by calculating separate EVM statistics for symbols 
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EMWIN 



Figure 2 Measured BER and curve fitted BER 

sampled immediately after a transition, and symbols sampled without a change in logical value 
as shown in Figure 3. This only removes channel memory effects due to the previous symbol, 

which provided sufficient separation for 
our purposes. There are two ways to 
determine a and P: 

1 . Use only the inner blobs (which 
contributes the most to errors) 

2. Find the EVM for the inner and outer 
blobs and average the two EVM 
measurements 


Three methods of calibrating EVM to predict BER 
by determining the two coefficients were 
investigated; these were no added noise at RF, no 
added noise at IF and added noise at RF. Our set- 
up allowed simultaneous EVM and BER 
measurements to remove any potential inaccuracies 
due to variation with time. 

Figure 3 ISI separated by symbol sequence * The n ° added noise method uses a single 

EVM measurement at RF with a very high 

S/N and adjusts the coefficients for best fit to the BER curve. 

• The no added noise at IF method used the same method as above, but the EVM measurement is at IF so any 
RF to IF distortion is captured. 
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• The added noise method used simultaneous multiple RF EVM measurements and BER measurements with 
varying amounts of added noise with a and P varied to obtain the best fit. 

For reliability the satellite communications subsystem has duplicate hardware for each service and are called the 
primary and the redundant. To compare the effectiveness of the three methods, calibration based on the primary 
side of the communications subsystem was used to predict the BER from the redundant side for each link at the 
spacecraft level. The redundant sides generally had a slightly different curve; for example, the redundant side of the 
EMWIN link in Figure 2 had a 0.35 dB greater implementation margin than the primary side. As seen, EVM 
accurately predicted the difference in BER curves. 

The results of the comparison of the three methods for the eight links tested are shown in Figure 4. All results were 
in agreement with measured values to within 0.5 dB. The RF no added noise method, besides being much quicker, 
proved slightly more accurate than the other two methods. With one exception, the predictions based on the RF, no 
added noise method agreed with measurements to with 0.2 dB which is about the accuracy of the calibration and the 
BER measurements. We had expected that the added noise would be the best method because the additional 
measurements would provide more data points to fit the BER curve. Second best method was expected to be the IF 
no-added noise measurements as these would exclude additional distortion in the RF chain. The RF, no added noise 
was expected to be the poorest method. Instead, the results showed the opposite was true, the RF no added noise 
was best, the IF no added noise and finally, RF with added noise was the worst. One possible reason for this is that 
adding noise also adds another source of measurement error that is not present with the no added noise methods. 

For integration and test, the RF no added noise method was used as this method was the simplest, and most accurate. 
However EVM measurements were made with added noise at the same time the BER measurements were made. 

The latter were used in post-launch testing described later in this paper. 

Table 1 EVM measurements with and without adjacent channel interference 

In addition to saving time, EVM 
allowed measurements that 
would have been impossible 
with traditional BER methods 
which use pseudo-random 
number sequence. EVM allowed 
measurements of actual data 
with non pseudo-random 
structure including headers from 
instrument data packets on the 
satellite. As described in an 
adjacent paper 6 , EVM can 
identify the source of 
degradations, including inter- 
symbol interference and phase 

EVM cannot replace BER measurements for two reasons. First, a calibration against a BER test set is required. 
Second, if there is a large change in EVM due to a new, non-thermal degradation, EVM cannot accurately predict 
the BER. As an example, a large increase in phase noise (comparable to the thermal noise) would invalidate the 
calibration. The phase noise would be immediately apparent from a constellation plot. EVM would in this case 
detect the change in performance, but not accurately predict the new BER. A new calibration curve would be 
necessary to predict BER. 

More commonly, EVM will vary over a narrow range, perhaps due to thermal variations. The methods described 
here allow a determination of the significance of that variation. 

An example of the utility of EVM is determining if adjacent channel interference (ACI) degrades link quality. EVM 
measurements were made both with only the channel under test on and with all channels on with modulation. As 
shown in Table 1, the agreement of both methods showed that there was no degradation due to ACI. 


Percent EVM 

Service 

GOES-P 

GOES-O 

PMT No 
ACI 

PMT with 
ACI 

AMB1 No 
ACI 

with ACI 

DCPI01 

1 .65% 

2.20% 

2.16% 

2.29% 

DCPR01 

2.75% 

2.85% 

2.37% 

2.34% 

EMWIN01 

3.25% 

3.40% 

3.61% 

3.00% 

MDL01 

3.70% 

3.75% 

3.52% 

3.49% 

PDR01 

4.40% 

4.50% 

4.75% 

5.08% 

SAR04 

4.15% 

4.25% 

4.06% 

4.22% 

SD02 

3.45% 

3.45% 

4.78% 

4.93% 

WEFAX01 

10.25% 

10.55% 

9.17% 

9.22% 


noise. 
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The benefits of EVM carried over to the post launch testing (PLT) which verified the usefulness of EVM 
measurements for PLT. Very good agreement with ground testing was obtained and will be discussed 

IV. Test Setup & Measurement Repeatability 

From the past experiences in BER measurements, it was found that the average measurement repeatability of BER is 
around +/- 0.3 dB and this accuracy is very difficult to maintain due to the difficulty of calibrating the test set-up. 
The conventional BER test on a spacecraft requires a modulator, up & down converters (in most of cases), noise 
sources, additional filters, and a receiver. The EVM method introduced in this paper does not require noise sources, 
additional filters and a receiver to track performance. Furthermore, the latest VSA models support even higher 
frequencies up to X-band without requiring a downconverter. This eliminates a lot of the efforts required by BER 
measurement and increases measurement consistency significantly. The purpose of BER measurement in PLT is not 
only to perform the BER test in an on-orbit configuration but also to compare the result with the ground test results 
performed at the contractor’s facilities, thus establishing a baseline to track satellite performance over life. 

EVM is the only method to track the quality of service for the uncoded science data generated onboard the 
spacecraft. In commercial communications satellites, EVM is the only method of determining the quality of service 
of links where the link is uncoded or the coding is not known and there is no access to the uplink data. 

One surprising aspect of VSAs is that they do not process data in the same way. One type of VSA finds the standard 
deviation of EVM for 10,000 points. To get the value for 60,000 points, 6 measurements of 10,000 points each are 
taken and the 6 are averaged. This gives a different (and incorrect) answer from another type of VSA which 
correctly finds the standard deviation for all 60,000 points. 

EVM Methods Compared 
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Figure 4 Three EVM calibration methods compared 


V. Analysis Methodology 

A one-time calibration of the EVM measurements is made by conducting simultaneous EVM and BER 
measurements at the start of Integration and Test (I&T). There are two steps to this calibration, as the direct BER 
measurements are over a smaller Eb/No range (due to specification needs and time constraints). The first step is to 
measure BER and EVM over the range of Eb/No, and continue to measure EVM over a larger range of Eb/No. The 

5 

American Institute of Aeronautics and Astronautics 


added noise BER curve from EVM (a2 and P2) is generated. The second step is to generate a BER curve (al and 
pi) from the no-added-noise EVM to the curve generated in the first step. Finally the no added noise BER curve is 
compared to the original BER measurement. Both sets of coefficients, a and P are calculated using EXCEL’s Solver 
to vary a and P to find the minimum of the sum of the squares of the difference between the measured and predicted 
BER, as shown in Table 2. The starting points used by Solver sometimes must be varied to end up with the best 
curve. 

The al and pi coefficients are used through the rest of the test program with the no added noise EVM measurement 
to generate the BER curves. 


Table 2 Obtaining Calibration Coefficients 
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al 

2.2049 

a2 

0.1298 
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PI 

0.61901117 

P2 

0.424287703 
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Measured BER 

EVM Calibration Example 

Normalized 
deviation 
for No 
Added 
Noise Case 

Normalized 
deviation 
for Added 
Noise case 

6 

Calculated BER 
from - No Noise 
Added Case 

BER Curve fit from measured 
BER and EVM - Added Noise Case 

7 

9 

Eb/No 

BIT 

ERROR 

RATE(Pe) 

EVM 

(%) 

BER (Pe) 

Eb/No 

EVM (%) 

BER (Pe) 



10 

10.035 

4.02E-04 

6.52 

2.973E-04 

10.035 

30.94 

3.001E-04 

9.28E-10 

3.05E-01 

11 

11.035 

5.96E-05 

6.52 

6.332E-05 

11.035 

27.70 

6.351E-05 

2.49E-02 

2.49E-02 

12 

12.035 

9.26E-06 

6.52 

1.039E-05 

12.035 

25.11 

1.061E-05 

-1.41E-01 

-1.41E-01 

13 

13.035 

1.26E-06 

6.52 

1.290E-06 

13.035 

22.75 

1.306E-06 

-7.28E-02 

-7.28E-02 

14 

14.035 

1.29E-07 

6.52 

1.203E-07 

14.035 

20.67 

1.152E-07 

6.82E-02 

4.65E-03 
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VII Post Launch Testing with EVM 

The coefficient sets calibrated on the GOES-O & P spacecrafts for each service had to be used for GOES-N, (which 
was renamed GOES 13 once in service) PLT as there was no ground EVM calibration data for GOES-13. Despite 
this, the measured bit error rates trended well with BER predicted from EVM measurements, within a few tenths of 
a dB. 

There is no way to take a noise free measurement of EVM in PLT as the link adds a substantial amount of noise. 
Therefore we used the coefficients derived from the added noise EVM measurements to generate a BER vs. Eb/No 
curve. From a single EVM measurement and knowing Eb/No, a BER vs. Eb/No was generated. From this curve, 
the BER at the specified Eb/No was found and specification performance verified. The bit error rate estimation 
from the EVM measurement in this example clearly shows this method can closely and consistently estimate the bit 
error rate results within a bit error rate measurement repeatability and uncertainty boundaries. For one way links, e.g. 
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links where a varying data pattern is generated on the spacecraft , the only way to get BER performance is with 
EVM as there was no way to have knowledge of the flight bit pattern. 


GOES - BER Ground Measured and predicted BER from PLT EVM measurements using GOES O 

and P coefficients 



— Theory BER 


-♦-GOES N BER from 
GOES-O Coef 


-B-GOESN BER from 
GOES-P Coef 


— ♦ — GOES-N Measured 
BER (Pre SCTV) 


x GOES-N Measured 
BER (RFA2) 


Figure 5 BER trending on GOES 13 using coefficients on GOES O and P 

VII. Conclusions 

While EVM provides a very simple and quick method for estimating the bit error rate of a communication channel, 
there are several limitations need to be considered carefully during the calibration and the measurement. 

• Changing any hardware after the point at which EVM is measured could invalidate the calibration since 
EVM measures the quality of the link up to the point where the VSA is connected and not past that point. 

• If there are substantial changes in the EVM values, EVM may not accurately predict the new BER based on 
the prior calibration. For example, if phase noise suddenly degrades performance EVM increases 
dramatically and so will the link degradation. The new measure must be evaluated based on the range of 
the calibration. But either way, a big variation from expected EVM should lead to more investigation. 

• Vector Signal Analyzer setup: Use consistent instrument setup parameters. The instrument setup 
parameters should be well maintained and controlled during test phases to avoid any possible measurement 
errors introduced by inconsistent configuration of the instrument. We recorded VSA setup parameters for 
each modulation format on a disk and used the same settings throughout the test program. 

• EVM can not detect errors that occur outside of the short measurement time (infrequent burst errors). 

• For PLT or other applications where there is a substantial contribution from thermal noise, the noise added 
method should be used. Where Eb/No is high, the no added noise case should be used. 

• If the two blob effect (ISI) is seen in the VSA, the EVM data must be processed to separate out the EVM 
for each blob. 
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